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ABSTRACT 



Different forms of accommodation have been suggested in the 
assessment of English language learners (ELL) . This study examined the 
effectiveness of a few accommodation strategies on the performance of ELL 
students in mathematics. A group of 946 eighth graders participated in this 
study. Using mathematics test items from the National Assessment of 
Educational Progress, researchers used four accommodation strategies: (1) 
modified (simplified) English language of the test items; (2) a glossary; (3) 
the original English with extra test time; and (4) the glossary with extra 
time. Students were assigned randomly to the different accommodation forms 
and to a comparison group within participating classrooms to control for 
teacher and school effects. The results of the analyses suggest that 
receiving extra time may have an impact on students' mathematics performance. 
Students, particularly ELLs, performed slightly higher on the modified 
version. When students received extra time to work on their mathematics 
tests, their scores increased about a point, indicating a potential 
accommodation effect. It appears that the presence of the glossary on 
nonmathematics-related terms may have had minimal effect on students' 
mathematics performance. However, when the glossary was combined with 
receiving extra time, the mathematics performance among the students appears 
to be the greatest . (Author/SLD) 
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Abstract 

Different forms of accommodations have been suggested in the 
assessment of English language learners. This study examined the 
effectiveness of a few accommodation strategies on the performance of ELL 
students in math. 

A group of 946 8th-grade students participated in this study. Using 
mathematics test items from the National Assessment of Educational 
Progress (NAEP), we employed four accommodation strategies: 1) Modified 
(simplified! English language of the test items (used for the first time here as a 
form of accommodation for limited English proficient students); 2) Glossary: 

3) Original English with extra time: and 4) Glossary with extra time . Students 
were assigned randomly to the different accommodation forms and to a 
comparison group within participating classrooms, to control for teacher and 
school effects. 



The results of the analyses suggest that receiving extra time may impact 
students' math performance. Students, particularly ELLs performed slightly 
higher on the modified version. When students received extra time to work 
on their math tests, their scores increased about a point, indicating a potential 
accommodation effect. It appears that the presence of the glossary of non- 
math related terms may have had minimal effect on students' math 
performance. However, when the glossary was combined with receiving 



1 For a detailed description of the study design, findings, and interpretations, see Abedi, Hofstetter, and 
Lord, 1998. 



o 

UMi BEST COPY AVAILABLE 3 



Differential Impact of Accommodation 2 



extra time, the math performance among the students appears to be the 
greatest 



Perspective 

Research suggests that content-based knowledge among linguistic 
minority students can be underestimated if the student is not proficient in the 
language of instruction and assessment (Abedi, Lord, and Plummer, 1995; 
Abedi, Lord, and Hofstetter, 1998; Alderman, 1981; Garcia, 1991; LaCelle- 
Peterson & Rivera, 1994). Recent federal and state legislation, including the 
reauthorization of the Elementary and Secondary Education Act (ESEA), 
through the enactment of the Improving America's Schools Act (IASA) of 
1994, now state that all children should be given educational experiences to 
assist them in achieving high standards. This implies that children 
previously excluded from assessments because of physical or psychological 
disability or because of limited proficiency in English must have the 
opportunity to participate in these assessments (LaCelle-Peterson & Rivera, 
1994; Zehler, Hopstock, Fleischman, & Greniuk, 1994; August & Hakuta, 
1997). 

However, achieving the goal of increased inclusion in large-scale 
assessments requires a complex set of practical and technical decisions. The 
literature suggests different forms of accommodations for students with 
limited English proficiency to help increase their participation in assessment. 
Over half of the states (55%) permit accommodations for limited English 
proficient (LEP) students. Among the accommodations used by states are: (1) 
accommodations related to timing, such as allowing extended time and 
providing more breaks during the test session; (2) accommodations in 
assessment environment, such as taking the test alone or at home; (3) 
modifications of response format, such as giving oral responses or responding 
in sign language; and (4) modifications of the presentation format such as 
reading the directions out loud or giving directions in sign language (see 
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Olson & Goldstein, 1997; Hafner, 1995; also, Council of Chief State School 
Officers Sc North Central Regional Educational Laboratory, 1996). 

This study was the first to employ linguistic clarification of test items as 
a form of accommodation in comparison with other adaptations for English 
language learners. Unlike most other forms of accommodation, this form, if 
shown to be effective, is feasible and easy to implement. It does not create a 
burden for the test administrators or test takers. The linguistic modification 
of test items can be done at the item-writing stage; it can help all test takers, 
particularly those with limited English proficiency. In linguistically modified 
selected NAEP math items, Abedi, Lord and Plummer (1995) found modest 
but significant effects among 8th grade students with lower levels of English 
proficiency and with students enrolled in lower levels of mathematics classes. 
A follow-up study yielded similar results (Abedi, Lord and Hofstetter, 1998). 
Abedi et al. (1998) also found that while clarifying the language of math test 
items helped all students improve their performance, in 34 percent of the 
items for which a modified version was created, LEP students benefited more 
than non-LEP students. Further, certain types of items may have contributed 
more than others to the significant math score differences. 2 

This study examined the effects of students' background characteristics 
on the degree of impact of accommodations for LEP students on their 
performance in content-area assessments. 

Method 

Participants 

Data were collected from 946 8th-grade students (ages 13-14) (see Abedi, 
Hofstetter, Baker, & Lord, 1998 for a detailed description of the sample) from 
32 math classrooms in 5 middle schools in southern California. Each student 
completed a questionnaire providing language background information. 

Although the broader term, "English language learner" (ELL) avoids the negative connotation of 
the term "iimited English proficient" (LEP) (LaCelle-Peterson & Rivera, 1994; Butler & Stevens, 1997), we 
use the term LEP here to reflect classification by schools according to criteria for participation in NAEP 
and government-funded programs. 
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Table 1 presents some of the background characteristics of the subjects in the 
study. As the data in Table 1 indicate, nearly three-quarters of the students 
(72%) reported their ethnicity as Hispanic, while the remaining described 
themselves as Asian or Pacific Islander (14.7%), White (6%), African- 
American (5.3%), or Other (1.4%). A majority of the students reported that 
they were from the United States (57.1%), followed by Mexico (23.4%). 
Among those who indicated that they spoke another language besides English 
(85.1%), Spanish was the most commonly listed (82%). Over half of the 
students were designated by their schools as limited English proficient 
(52.8%), while the others had transitioned into non-LEP programs and were 
designated Fluent English Proficient (FEP-30.4%), or were Initially Fluent in 
English (IFE-16.8%). 



Table 1 about here 
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Design 

In this study, four different accommodation strategies were used: 1) 
Modified (simplified’) English language of the test items, where items were 
rewritten to simplify potentially difficult vocabulary and linguistic structures 
but mathematics vocabulary was not changed (for a detail description of 
linguistic modification of test items, see Abedi, Lord, and Plummer, 1995); 2) 
Glossary, where non-math words or phrases identified as potentially difficult 
for LEP students to understand were defined or paraphrased; 3) English with 
extra time, where students were given an extra 25 minutes to work on the 
math test; and 4) Glossary with extra time, where students were given the 
glossary plus an extra 25 minutes. One test booklet was developed for each of 
the four forms of accommodation. To create a control or comparison group, a 
fifth booklet was added which included the original English version of the 
NAEP math items. Students were assigned randomly to the five different 
booklets within participating classrooms, to control for teacher and school 
effects. 

Items from the 1996 NAEP Grade 8 Bilingual Mathematics booklet were 
used. To obtain a measure of students' language proficiency, a NAEP reading 
test was also used. Students read a 2-page story in English, then responded to 
11 questions (7 selected response, 4 constructed response). The passage and 
items were from a secured 1992 Grade 8 Reading assessment. 

In addition to the math and reading tests, each student was also 
administered a 45-item questionnaire, comprising primarily items from the 
1996 NAEP Grade 8 Bilingual Mathematics booklet, relating to students' 
attitudes toward mathematics, grades in mathematics, self-reports of ability to 
understand math terminology and in performing computations, and 
educational and mathematical ambitions. This questionnaire contained 
additional questions from an earlier language background study (Abedi, Lord, 
& Plummer, 1995). 
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Categorization of students into various student designations (students 
with limited English proficiency, LEP; initially fluent in English, IFE; and 
fluent English proficient, FEP) was obtained from the participating schools. 
Designations were based primarily on students' performance on English 
language proficiency tests administered at the schools upon entrance into the 
educational program, and were updated periodically. 

Overall Results of Accommodations 
Math Performance by Accommodation 

The results of analyses suggest that test accommodations affect 
students' test performance. Table 2 presents means and standard deviations 
of math test scores by students' LEP status and forms of accommodation. As 
the data in Table 2 indicate, for the entire sample, students who received the 
standard (original) English math test had a mean math score of 14.68 
(SD=6.67), out of 35 points possible. Linguistic modification (M=14.23, SD=6.3) 
and presence of a glossary of non-technical terms (M=14.53, SD=7.01) appeared 
to make no overall notable difference in student performance. However, the 
data suggest that extra time increased students' scores by one point (M=15.64, 
SD=6.86). Further, the data suggest that students who received the glossary in 
addition to extra time scored the highest overall (M=17.08, SD=7.68). These 
students had math scores approximately 2 & half-point (0.36 standard 
deviation) higher than students who received no accommodation at all. 



Table 2 about here 



Accommodation effects may also be examined by comparing math 
performance by LEP status. LEP students performed lower (M=12.30, SD=5.67) 
than their more English fluent counterparts (M=17.45, SD=6.83) — a 
difference of over 5 points. This trend was maintained across test booklets. 
For example, LEP students who received the standard math assessment 
(original English) reported a mean score of 12.07 (SD=5.47), while FEP/IFE 
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students had a mean score of 17.56 (SD=6.70) Interestingly, linguistic 
modification appeared to aid LEP students (M=12.63. 5D=5.23 as compared 
with M=12.07, SD=5.47 for the original version). 

In comparison with the standard test (original items), LEP students' 
scores were lower on the same items when a glossary was provided (M=11.84, 
SD=5.94), perhaps because of information overload, while scores for FEP/IFE 
students with the glossary increased (M=17.78, SD=6.84). Extra time appeared 
to help all students, both LEP and FEP/IFE. LEP student scores increased 
slightly with extra time (M=12.93, SD=5.99), and even more when they 
received the glossary with extra time (M=13.69, SD=6.74). For FEP/IFE 
students, extra time alone increased math scores by more than one point 
(M=18.88, SD=6.50), and the addition of a glossary resulted in almost a 3-point 
gain (M=20.37, SD=7.17). Overall, these results suggest that the linguistic 
modification may help LEP students, as a possible accommodation. Further, 
all students benefited from extra time and glossary. These trends remained 
stable, even after controlling for the students' reading achievement scores. 

The results of analysis of variance comparing students across the 
different forms of accommodation and LEP status indicated that student 
performance across the categories of accommodations differs significantly, F 
(4, 886) = 2.71; p=0.029. Similarly, students performed differently across the 
LEP categories, F (1,886) = 103.67; p < .01). As indicated earlier, students with 
limited English proficiency performed lower than the native English 
speakers. 

Results of Overall Reading Performance 

The reading test, from the NAEP Grade 8 reading assessment, was 
administered to obtain a measure of the students' reading proficiency. 
Because of time constraints in the testing environment, a single section was 
selected with only one reading passage and 11 responses. 




Differential Impact of Accommodation 8 



Table 3 summarizes the results of descriptive analyses of the reading test. 
As the data in Table 3 indicate, overall, the mean reading test scores were 
fairly low (M=5.07, SD=3.22, n=896). The most notable finding is the 
difference between the LEP and non-LEP students' performance on the 
reading assessment. As expected, FEP/IFE students (M=6.35, SD=3.12, n=423) 
consistently performed higher on the reading test than LEP students (M=3.92, 
SD=2.86, n=473) — an approximate two & half-point difference, which was 
statistically significant, F (1,886) = 79.49; p< .01. 



Table 3 about here 



This finding provides evidence that the reading achievement test, 
despite its limitations related to validity and adequacy as a measure of 
students' reading proficiency, emerged as a suitable predictor of math 
performance. FEP/IFE students scored higher on reading tests and math tests. 
Further, students with a better command of English text (FEP/IFE students) 
were likely more able to read and interpret the math items correctly than 
students with lower English proficiency levels (LEP students). 

As the reading test was the same for all students, regardless of test 
booklet, we would expect the reading scores to be comparable across test 
booklet groups. However, the score means in Table 3 suggest that students 
receiving the "Modified English" test booklet scored lower than students 
receiving other test booklets. 

Among LEP students, the groups that were given the Original English 
and Modified English booklets showed no significant difference between their 
scores on the (identical) reading tests (means were 3.78 and 3.84, respectively). 
However, among the FEP/IFE students, the groups given the Original English 
and Modified English booklets scored 6.77 and 5.81, respectively on the 
reading test—nearly a one-point difference. This difference in reading ability 
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might have contributed to the lower math score on the modified English 
booklet for the FEP/IFE students. Note that, as Table 2 shows, the FEP/IFE 
students scored lower on the modified English math test. Among the 
FEP/IFE students. Table 3 shows, the group with the highest reading score was 
the group that answered the English math items. 

Discussion 

In this study, four different forms of accommodation were used in 
assessing 8th grade students' mathematics performance. Among the 
accommodation strategies employed in this study was the language 
clarification of test items, used as form of accommodation for English 
language learners for the first time in the literature of accommodation. The 
findings of this study suggested that different forms of accommodation 
produced different results. In this study, the form of accommodation with 
the greatest effect on all students was the provision of an English glossary 
with definitions or paraphrases of potentially difficult non-mathematical 
words or phrases plus extra time. It is important to note that the English 
glossary alone (that is, without extra time) did not help the students in this 
study and in some cases even had a negative impact on student performance. 
This, we suggest, may be due to information overload. That is, students were 
presented with more data than they could process in the limited amount of 
time that they had to answer the test items. 

One of the most important findings of this study is that some forms of 
accommodation may help all students, improving students' performance 
across all subgroups. This may not be a desirable outcome, since it may 
impact the validity of the accommodation and may affect the construct under 
measurement (mathematics understanding in this study). 

The provision of Extra Time Only resulted in slightly higher math scores 
for most students but not for all subgroups; extra time did not produce higher 
scores for students in lower level math classes (8th grade math, as opposed to 
pre-algebra and algebra classes). 
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Among the various forms of accommodation used in this study, the 
language clarification of test items was the only one, which helped LEP 
students more than non-LEP students. This difference, though small, 
suggests that we give further attention to the linguistic modification of test 
items as a form of accommodation which is less expensive and more feasible 
logistically than other forms of accommodation for English language learners. 
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Table 1. Background Characteristics of Students Participating in this Study- 



Variable name 


Frequency 


Percent 


Cum. Per 


Gender 








Male 


455 


50.1 


50.1 


Female 


453 


49.9 


100.0 


Ethnicity 








White (not Hispanic) 


54 


6.0 


6.0 


African American 


48 


5.3 


11.3 


Hispanic 


650 


72.0 


83.3 


Asian or Pacific Islander 


133 


14.7 


98.0 


American Indian-Alaskan 


5 


0.6 


98.6 


Other 


13 


1.4 


100.0 


Speak Other Language? 








Yes 


773 


85.1 


85.1 


No 


135 


14.9 


100.0 


LEP Status 








Limited English Proficient (LEP) 


473 


52.8 


52.8 


Fluent English Proficient (FEP) 


272 


30.4 


83.1 


Initially Fluent in English (IFE) 


151 


16.8 


100.0 


Language of Instruction 








English Only 


723 


76.4 


76.4 


Spanish Only 


18 


1.9 


78.3 


English Shelter 


205 


21.7 


100.0 


Country of Origin 








United States 


540 


57.1 


57.1 


Mexico 


221 


23.4 


80.4 


Other Countries 


185 


19.5 


100.0 


Studied Math in Other Language? 








Yes 


512 


59.4 


59.4 


No 


350 


40.6 


100.0 


Number of Years Lived in U.S. 








1-3 


69 


7.5 


7.5 


4-6 


76 


8.2 


15.7 


7-9 


121 


13.0 


28.7 


10-12 


59 


6.4 


35.1 


More than 12 years 


607 


65.0 


100.0 


Kind of Math Class Taking This Year 








Not Taking Math 


12 


1.4 


1.4 


8th-Grade Math 


336 


38.0 


39.3 


Prealgebra 


215 


24.3 


63.6 


Algebra 


244 


27.6 


91.1 


Other 


78 


8.9 


100.0 



Note: Totals do not add up to 946 due to missing data 
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Table 2. 

Mean NAEP Math Achievement Scores for 8th Grade Students (35 points 
possible) 



Math Book 




LEP Status 




LEP 


FEP/IFE 


COLUMN 

AVERAGE 


Original English 


12.07 


17.56 


14.68 




(SD=5.47; n=144) 


(SD=6.70; n=130) 


(SD=6.67; n=274) 


Modified English 


12.63 


15.94 


14.23 




(SD=5.23; n=124) 


(SD=6.67; n=117) 


(SD=6.19; n=241) 


Glossary only 


11.84 


17.78 


14.53 




(SD=5.94; n=146) 


(SD=6.84; n=121) 


(SD=7.01; n=267) 


Extra Time only 


12.93 


18.88 


15.64 




(SD=5.99; n=30) 


(SD=6.50; n=25) 


(SD=6.86; n=55) 


Glossary 


13.69 


20.37 


17.08 


+Extra Time 


(SD=6.74; n=29) 


(SD=7.17; n=30) 


(SD=7.68; n=59) 




12.30 


17.45 


14.73 


ROW AVERAGE 


(SD=5.67; n=473) 


(SD=6.83; n=423) 


(SD=6.75; n=896) 



Table 3. 

Mean NAEP Reading Achievement Scores for 8th Grade Students (11 points 
possible) 



LEP Status 

Math Book 

LEP FEP/IFE COLUMN TOTAL 



Original English 


3.78 


6.77 


5.20 




(SD=2.80; n=144) 


(SD=2.91; n=130) 


(SD=3.22; n=274) 


Modified English 


3.84 


5.81 


4.80 




(SD=2.91; n=124) 


(SD=3.26; n=H7) 


(SD=3.23; n=241) 


Glossary 


4.01 


6.50 


5.13 




(SD=2.92; n=146) 


(SD=3.01; n=121) 


(SD=3.21; n=26 7) 


Extra Time 


3.93 


6.40 


5.05 




(SD=2.69; n=30) 


(SD=3.34; n=25) 


(SD=3.22; n=55) 


Glossary 


4.48 


6.10 


5.31 


+ Extra Time 


(SD=2.87; n=29) 


(SD=3.61; n=30) 


(SD=3.34; n=59) 




3.92 


6.35 


5.07 


ROW TOTAL 


(SD=2.86; n=473) 


(SD=3.12; n=423) 


(SD=3.22; n=896) 
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